This invention relates to electronic computation networks, and more particularly, to highly parallel associative computation networks that are also known as neural networks.
Electronic neural networks have been described by J. J. Hopfield in "Neurons With Graded Response Have Collective Computational Properties Like Those of Two-state Neurons", Proc. Natl. Sci., USA Vol. 81, pp. 3088-3092; and by J. J. Hopfield and D. W. Tank in "`Neural` Computation of Decisions in Optimization Problems", Biological Cybernetics, Vol. 52, (1985), pp. 141-152; as well as in U.S. Pat. No. 4,719,591 issued Jan. 12, 1988, J. J. Hopfield and D. W. Tank and U.S. Pat. No. 4,660,166 issued Apr. 21, 1987, J. J. Hopfield.
Basically, the Hopfield neural network described in the above references is a highly parallel computational circuit comprising a plurality of amplifiers, with each of the amplifiers feeding back its output signal to itself and all of the other amplifiers through conductance T.sub.ij. The T.sub.ij conductances (where T.sub.ij denotes the conductance between the output of amplifier j and the input of amplifier i) and the associated connections can be thought of as comprising a feedback network which has one output signal set and two input signal sets. The output signal set is applied to the amplifier inputs, one of the input signal sets is derived from the amplifier outputs, and the other input signal set is responsive to input stimuli applied to the neural network. As shown in the prior art, one can explicitly specify the values of the T.sub.ij conductances to achieve predetermined results, such as reaching different specified output states of the amplifier in response to different ranges of input stimuli. Also as described in the prior art, an input interconnection network may be interposed between the input stimuli and the second set of inputs of the feedback network. The input interconnection network permits manipulation of the expected input signals to corresponding signals that drive the feedback network and the amplifiers.
The neural network model described most extensively in the prior art is one with symmetric couplings, i.e., the connections between pairs of neurons satisfy the relationship T.sub.ij =T.sub.ji. The dynamics of such a network is relatively simple. The system moves in the direction of reducing a global energy function of the circuit, E, to states that are local minima of E, and once a local minimum is reached, the circuit remains at the stable state until perturbed by a sufficiently large input signal that moves the circuit to a different local minimum. The local minima may be thought of as the stored "memories" that are characterized by the vectors M.sup..nu.. An associative memory can be constructed with the Hopfield neural network by constructing the connection strengths T.sub.ij in accordance with the outer product, or Hebb, rule; to wit, by assigning ##EQU1## for j.noteq.i, and 0 otherwise. In a very simple embodiment T.sub.ij =-1 (an inhibitory connection) for i.noteq.j and 0 otherwise, resulting in a network that behaves as an n-flop; that is, a multi-output version of the well-known, two-output network commonly known as a flip-flop.
The Hopfield neural network is a very robust network, in the sense that it is able to associatively recall the stored memories even when the input is heavily distorted by noise, or when some of the hardware connections are inoperative. On the other hand, the Hopfield network must be sparsely populated with stored information, and some work has been done which indicates that the number of stored vectors in a Hopfield network should be limited to approximately 0.15N, where N is the number of amplifiers. Also, computation of the T.sub.ij values in the Hopfield network (for the general case) is not easy, and the precise location of the boundaries between regions of attraction of the memories is not easily controlled.
It is an object of this invention to provide a network structure that allows for simple and efficient programming of the memories into the network.
It is another object of this invention to provide an arrangement that can easily be implemented in integrated circuit technology.